Skip to content

perf: optimize regexp_instr (40% faster)#23540

Open
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/regexp_instr-datafusion-20260713-155538
Open

perf: optimize regexp_instr (40% faster)#23540
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/regexp_instr-datafusion-20260713-155538

Conversation

@andygrove

@andygrove andygrove commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A

Rationale for this change

Optimize existing expression.

What changes are included in this PR?

Removed per-batch default-array/Vec allocations, memoized the compiled regex across rows to skip HashMap hashing for literal patterns, and skipped the O(n) chars() scan on the default start=1 path.

Are these changes tested?

Existing tests.

Benchmark (criterion):

  • regexp_instr_with_start [size=1024, str_len=32]: 40.073% faster (base 41127ns -> cand 24646ns)
  • regexp_instr_with_start [size=1024, str_len=128]: 44.498% faster (base 48215ns -> cand 26760ns)
  • regexp_instr_no_start [size=1024, str_len=128]: 47.253% faster (base 49244ns -> cand 25974ns)
  • regexp_instr_no_start [size=1024, str_len=32]: 42.18% faster (base 40495ns -> cand 23414ns)

Full criterion output:

regexp_instr_no_start [size=1024, str_len=32]
                        time:   [23.389 µs 23.479 µs 23.645 µs]
                        change: [−42.300% −42.180% −42.004%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) low mild
  2 (2.00%) high severe

regexp_instr_with_start [size=1024, str_len=32]
                        time:   [24.617 µs 24.635 µs 24.659 µs]
                        change: [−40.118% −40.073% −40.026%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

regexp_instr_no_start [size=1024, str_len=128]
                        time:   [25.756 µs 25.791 µs 25.830 µs]
                        change: [−47.394% −47.253% −47.076%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

regexp_instr_with_start [size=1024, str_len=128]
                        time:   [26.645 µs 26.666 µs 26.693 µs]
                        change: [−44.577% −44.498% −44.417%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Are there any user-facing changes?

No

@github-actions github-actions Bot added the functions Changes to functions implementation label Jul 13, 2026
@andygrove andygrove changed the title perf: optimize regexp_instr in datafusion-functions perf: optimize regexp_instr (40% faster) Jul 14, 2026
@andygrove andygrove marked this pull request as ready for review July 14, 2026 15:57
comphead
comphead reviewed Jul 14, 2026
View reviewed changes
Comment thread datafusion/functions/src/regex/regexpinstr.rs
};
let pattern: &Regex = compile_and_cache_regex(pattern, flags, regex_cache)?;
// println!("get_index: value = {}, pattern = {}, start = {}, n = {}, subexpr = {}, flags = {:?}", value, pattern, start, n, subexpr, flags);
) -> Result<i64, ArrowError> {

@comphead comphead Jul 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@comphead comphead Jul 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it return -1 if there is no match? 0 sounds like legit index

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@comphead comphead comphead left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

functions Changes to functions implementation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@andygrove @comphead